Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support common google apis and fix issue with ";" in package names #26

Closed
wants to merge 2 commits into from
Closed

Conversation

li-peng-dd
Copy link

I noticed that if you try to start a service with a proto file that includes a dependency on something with package like go_package=github.com/golang/protobuf/protoc-gen-go/descriptor;descriptor, it fails with:

2019/09/05 03:16:17 Failed to generate server formatting 20:19: invalid import path: "github.com/golang/protobuf/protoc-gen-go/descriptor;descriptor" (and 1 more errors)

This is because server.tmpl just naively uses the package declaration as the import statement, which results in invalid like above. I fixed truncating the package declaration at the semicolon.

I also propose to automatically include common google apis as part of Proto Path as I find them very commonly used in services, so it would be a nice convenience feature.

@jekiapp
Copy link
Contributor

jekiapp commented Sep 6, 2019

so what does it mean with semicolon? does it propose us to use it as an alias?

@li-peng-dd
Copy link
Author

Yeah, it's basically an alias that allows you to specify a directory path that's different from the package name. See:

golang/protobuf#139
grpc-ecosystem/grpc-gateway#341

Not supporting it creates an issue when using standard protobuf dependencies like:

https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/descriptor.proto

@@ -169,6 +169,10 @@ func resolveDependencies(protos []*descriptor.FileDescriptorProto) map[string]st
continue
}

// some package declarations use a semicolon, which causes a formatting error
// ignore for now
pkg = strings.Split(pkg, ";")[0]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so here we should check wether it's contained a dedicated alias or not, if yes then use it as alias.

@jekiapp
Copy link
Contributor

jekiapp commented Sep 6, 2019

also please provide on how this has been tested. if possible update the unit test to cover this case.

@li-peng-dd
Copy link
Author

@jekiapp updated it to use alias instead

I tested by adding import "google/api/annotations.proto"; to example.proto,

docker build -t "li/gripmock" .
docker run -p 4770:4770 -p 4771:4771 -v <mypath>:/proto li/gripmock /proto/examples/pb/example.proto

And verified that the server behaves as expected. Without the change the server fails to start with the error specified above.

Unfortunately I'm pretty new to Go and I'm having trouble getting the tests to compile and run correctly, so haven't had luck with the unit test. If you can send me instructions to get my local build setup on a mac, I can give it another shot.

@jekiapp
Copy link
Contributor

jekiapp commented Sep 13, 2019

will test your PR today in my local

@@ -7,6 +7,7 @@ RUN mkdir /stubs
RUN apk -U --no-cache add git protobuf

RUN go get -u -v github.com/golang/protobuf/protoc-gen-go \
github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First I need to know whether this is a common package or not, otherwise I'll find a way how to add this kind of custom dependecies

Copy link
Author

@li-peng-dd li-peng-dd Sep 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option would be to get the same protos from this repo instead: https://github.com/googleapis/api-common-protos

These are pretty common protos advocated by Google as part of their API design guide, so it's used by a lot of folks.

A more generic solution would be to have an option to pass in a specific protodir (that's mounted on the image), instead of just defaulting the protodir to the parent directory of the mocked proto file, this lets users be able to pull in all the dependencies in their entire proto project if they want.

i.e. my folder structure is:

protos/
├── google/
│   ├── http.proto
├── identity/
│   ├── auth.proto

If I mocked auth.proto, gripmock wouldn't be able to find http.proto as it sets protodir to protos/identity/, excluding the google folder. But If I could manually set protodir to protos/, that will fix the issue and give a more flexible way for users to specify dependencies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just created pr #28 to cover this case. please check it out. with that, user should be able to derive their own dockerfile and install their own dependencies then pass the custom protos with --imports arg

@jekiapp
Copy link
Contributor

jekiapp commented Sep 17, 2019

Meanwhile I just merged the fix of the original issue of this PR, sory I didn't use your PR since I also add integration test on #27 to prove my solution (support ; in go package)

@li-peng-dd
Copy link
Author

@jekiapp thanks, looks good!

@jekiapp jekiapp closed this Sep 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants